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ABSTRACT 

Background Adverse drug events, the unintended and 
harmful effects of medications, are important outcome 
measures in health services research. Yet no universally 
accepted set of International Classification of Diseases 
(ICD) revision 10 codes or coding algorithms exists to 
ensure their consistent identification in administrative 
data. Our objective was to synthesize a comprehensive 
set of ICD-10 codes used to identify adverse drug 
events. 

Methods We developed a systematic search strategy 
and applied it to five electronic reference databases. 
We searched relevant medical journals, conference 
proceedings, electronic grey literature and bibliographies 
of relevant studies, and contacted content experts for 
unpublished studies. One author reviewed the titles and 
abstracts for inclusion and exclusion criteria. Two 
authors reviewed eligible full-text articles and abstracted 
data in duplicate. Data were synthesized in a qualitative 
manner. 

Results Of 4241 titles identified, 41 were included. We 
found a total of 827 ICD-10 codes that have been used in 
the medical literature to identify adverse drug events. The 
median number of codes used to search for adverse drug 
events was 190 (IQR 1 56-289) with a large degree of 
variability between studies in the numbers and types of 
codes used. Authors commonly used external injury 
(Y40.0-59.9) and disease manifestation codes. Only two 
papers reported on the sensitivity of their code set. 
Conclusions Substantial variability exists in the 
methods used to identify adverse drug events in 
administrative data. Our work may serve as a point of 
reference for future research and consensus building in 
this area. 



INTRODUCTION 

The use of prescribed medications has risen dra- 
matically in the past decades. 1 In 2008, over 76% 
of Americans older than 60 years reported ingest- 
ing two or more prescribed medications daily, and 
37% used five or more. 1 Patients who use indicated 
medications appropriately can expect to derive 
benefit. Yet, a significant proportion will experience 
adverse drug events, the unintended and harmful 
effects resulting from medication use that are asso- 
ciated with suboptimal patient outcomes and 
increased health services utilization. 2-7 Adverse 
drug reactions alone, a subset of adverse drug 
events that occur when drugs are used in thera- 
peutic doses, 8 cause 5-10% of acute care hospital 
admissions, 9-12 prolong hospital stays, 13 and may 
contribute to more potential years of life lost than 
all other injuries combined. 14 Efforts to optimize 
medication use and reduce adverse drug events are 
therefore a public health priority. 15 



Adverse drug events that are encountered in clin- 
ical practice may vary substantially from those 
observed in pre-market clinical trials. 16 Reasons 
include differences in patient populations, treatment 
indications (eg, off-label use), monitoring protocols, 
duration of drug exposure and compliance between 
the clinical practice setting and the controlled envir- 
onment of clinical trials. 6 16 17 Monitoring and 
evaluating health outcomes that are associated with 
the way medications are used in clinical practice is 
difficult, yet essential to understanding the ongoing 
safety and risk-benefit profiles of medications, and 
paramount to promoting their optimal use. 

Administrative databases, electronic health 
records and disease registries contain a plethora of 
health data that can be used to ascertain health out- 
comes in clinical practice. These data are generally 
inexpensive, readily accessible and have been col- 
lected without interfering in the delivery of care. 
Thus, data from these sources are more likely to 
reflect the outcomes experienced by patients in the 
real-world clinical practice setting than the research 
setting, provided that the outcomes are appropri- 
ately identified and coded. 16 

Administrative databases worldwide, including in 
the USA, increasingly use the International 
Classification of Diseases (ICD) revision 10 system 
to classify diagnostic, health services utilization and 
death data. The ICD-10 coding dictionary enables 
coders to document adverse drug events in three 
ways: (1) by documenting the medication that 
caused an adverse drug event using 'external injury 
cause codes' (ie, Y40.0-59.9); (2) by documenting 
diagnoses that may be caused by a drug using 
'disease manifestation codes' (eg, A04.7 
Clostridium difficile colitis); and (3) by clustering 
an external injury cause code indicating the 
drug-related etiology with a disease manifestation 
code indicating the patient's diagnosis. 18 Because a 
large number of disease manifestation codes exist 
that might be adverse drug event related (eg, gastric 
ulcer), variation exists among health researchers in 
the code sets and coding algorithms used to iden- 
tify adverse drug events coded in ICD-10. 

Our main objective was to synthesize a compre- 
hensive set of ICD-10 codes used by health 
researchers to identify adverse drug events. Our 
secondary objective was to identify studies with 
ICD-10 coding algorithms for adverse drug events. 

METHODS 

Data sources and searches 

This was a qualitative systematic review of the lit- 
erature. Ethics approval was not required because it 
did not involve the use of human subjects or 
medical records. 
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A professional librarian (MDW) and study author (CMH) 
developed a systematic search strategy that was adapted for, and 
applied to, the following electronic bibliographic databases: 
MEDLINE (1948-2011), EMBASE (1980-2011), International 
Pharmaceutical Abstracts ((IPA) 1970-2011), Web of Science 
(1980-2011), the Cochrane Database of Systematic Reviews 
(1993-2011) and the Cochrane Central Register of Controlled 
Trials ((CENTRAL) 1996-2011) (see supplementary appendix 
A, available online only, for Medline search). Our search strat- 
egy combined three concepts: the ICD coding system, adverse 
drug events, and health outcomes. We reviewed the scope notes 
for each search term in order to identify and incorporate previ- 
ous indexing terms, alternative keywords, and appropriate 
MeSH terms. No language filters were applied. 

We hand-searched the following medical journals for relevant 
studies and conference proceedings from 2000 onwards: the 
Milbank Quarterly, Health Technology Assessment Journal, 
Health Affairs, Medical Care, American Journal of Medical Care, 
and Quality & Safety in Health Care. We used 2000 as the start 
date as this was the year that ICD- 10 was introduced in Canada. 
We did not search for any additional conference proceedings 
not published in the above journals, as we thought it would be 
unlikely for code sets to be published in abstract format. We 
conducted an electronic grey literature search using the search 
engine Google with the same search terms that we used for our 
electronic bibliographic database searches. We hand-searched 
the bibliographies of all relevant articles. In 2012, we conducted 
periodic environmental scans of the literature for newly pub- 
lished studies using auto alerts from MEDLINE, EMBASE and 
IPA. We contacted content experts and authors of relevant 
studies for any additional studies and for clarifications about 
their methodology and code sets. 

Study selection 

We included all studies reporting the use of the ICD-10 coding 
system to identify adverse drug events in adult patients. Studies 



had to report the ICD-10 code set (our outcome measure) or 
coding algorithms used to search the administrative data. We 
excluded studies reporting only pediatric data, as common man- 
ifestations of adverse drug events vary between adults and chil- 
dren. In addition, pediatric adverse drug events are more 
commonly the result of dosing errors or unintentional toxic 
ingestions compared to adult adverse drug events. We also 
excluded studies using other coding systems, written in lan- 
guages other than English, French and German, reporting only 
adverse events to illicit drugs or intentional overdoses, and 
studies that we could not access. 

One study author (AK) screened all titles for potential eligibil- 
ity using predefined criteria. Any potentially relevant studies 
were retained for abstract review. Two study authors (AK and 
CMH) reviewed the abstracts of potentially relevant titles. If 
either or both of the authors felt the abstract was potentially 
relevant, the full text article was retrieved and reviewed inde- 
pendently by two authors (AK and CMH) for inclusion and 
exclusion criteria. All disagreements about study eligibility at the 
full text review stage were resolved by achieving consensus 
through discussion. 

Data extraction and quality assessment 

Two authors (AK and CMH) independently abstracted data 
from included studies using standardized and piloted abstraction 
forms. Any disagreements over data abstraction points were 
resolved by achieving consensus through discussion, after con- 
tacting study authors for clarification. Data abstractors were not 
blinded to authorship or journal. 

We are unaware of any validated quality assessment scales to 
measure the quality of non-comparator, retrospective 
population-based cohort studies. 19 Therefore, we adapted rele- 
vant quality-assessment criteria from the GRACE guidelines and 
the York Centre for Dissemination and Reviews that were 
intended for population-based comparative effectiveness studies 
and reviews of adverse effects (table l). 19 20 



Table 1 Quality assessment criteria adapted from York Centre of Dissemination Reviews and the GRACE quality assessment checklist for this 
review of non-comparator cohort studies 19 20 



1 . Was the primary outcome(s) defined in a manner that 
was independent of the code set? 



2. Were methods for identifying the appropriate ICD-10 
codes to reflect the primary outcome reported, and was 
the search comprehensive? 



. Did the authors provide data or reference other work to 
allow the reader to understand how well the primary 
outcome was ascertained within the same data 
source(s) using the ICD-10 code set they chose 
(ie, sensitivity, specificity of the code set)? 



4. Were analyses conducted to test assumptions about the 
causal link between drug exposure and the disease, 
and how this uncertainty may have influenced the 
study results? 



Yes 
No 



NR 
Yes 



No 



NR 
Yes 



No 

NR 
Yes 

No 



The primary outcome(s) was defined in a manner that was independent from the code set 

The primary outcome(s) was not defined in a manner that was independent from the code set (ie, the 

definition was based on the ICD-10 codes used for searching) 

Not reported 

The methods for identifying the code set were explicit and comprehensive (eg, through literature review 
or mapping of pharmacovigilance terms to ICD-10 codes). It is unlikely that significant gaps in the code 
set exist 

The methods for identifying the code set was not reported, and the code set is not likely to be 

comprehensive 

Not reported 

The primary outcome was validated based on medical chart abstraction with clear definitions (eg, a 
formal medical record review of a sample of charts was done with adjudication of the primary diagnosis 
by a committee and the code set had reasonable sensitivity and specificity for identifying the primary 
outcome), or the code set was validated by linking and comparing existing data from various sources to 
ensure consistency and accuracy (eg, prospective registry compared with administrative data). 
Alternatively, previous work validated the code set, and the code set was likely to identify the stated 
primary outcome 

No data were reported, and no other work referenced, to suggest that the code set adequately identified 
the primary outcome 
Not reported 

Analyses were reported to evaluate the impact of uncertain causality on the study results (eg, analyses to 
test the impact of including codes for diagnoses that are likely, but not exclusively drug-induced, ie, 
Clostridium difficile colitis), and may not have been cluster coded with external cause codes 
No analysis was done to test the assumptions about the causal link between drug exposure and the 
disease manifestation 



ICD, International Classification of Diseases. 
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Data synthesis and analysis 

We synthesized the data in a qualitative manner, with two 
authors (CMH and AK) reviewing all data extraction forms and 
re-reading primary manuscripts. A third author reviewed all 
tables and figures for accuracy (LR), and all authors subse- 
quently critically reviewed the manuscript for content and 
accuracy. We adapted causality ratings for individual ICD-10 
codes from two previous publications, and modified them by 
adding the rating 'unlikely' (U) for codes that other authors 
used to identify adverse drug events in the literature that we felt 
unlikely to have indicated an adverse drug event (table 2). 21 22 
We also added the rating 'vaccine' (V) for codes that were 
vaccine related. Two study authors (CMH and JS) independ- 
ently, and blinded to one another's ratings, assigned causality 
scores to ICD-10 codes without previously assigned causality 
ratings, and came to consensus through discussion about any 
disagreement. 

Descriptive statistics were provided as averages with 95% CI, 
or medians with IQR. We calculated the interrater agreement of 
causality ratings assigned to ICD-10 codes, by collapsing causal- 
ity ratings into a category indicating that an adverse drug event 
was very likely (categories Al, A2, Bl, B2 and C), and a cat- 
egory in which adverse drugs events were deemed unlikely (cat- 
egories D, E, U and V) based on the previous literature. 21 22 We 
calculated k scores with 95% CI as a measure of agreement 
beyond chance alone. 

RESULTS 

Study characteristics and study quality 

Our search revealed 4241 titles, of which 41 met our inclusion 
and exclusion criteria (figure 1). Sixteen studies were conducted 
in Europe, 21-36 13 in North America, 14 37 ^ 8 nine in 
Australia, 49-57 and three in Asia (tables 3 and 4). 58-60 The 
majority of included studies were non-comparator retrospective 
studies that used administrative data to ascertain the prevalence 
of adverse drug events in population-based cohorts. 
Twenty-eight studies examined adverse drug events in general as 
the main outcome measure, 14 21 " 26 29 32 ^° 42 43 49 " 54 56 " 58 and 
13 examined drug or drug class-specific adverse drug 



events. 



27 28 30 31 41 44-48 55 59 60 



Eleven of 28 studies reported 
explicit methods for identifying the ICD-10 code set they 
used 21-23 34-36 38 40 42 53 58 

We found a total of 827 individual ICD-10 codes that have 
been used in the health literature to identify adverse drug events 
(see supplementary appendix B, available online only, for the 
complete list of codes). Of these, 175 were external injury cause 
codes (Y40.0-59.9), and 652 disease manifestation codes. Only 
13 disease manifestation codes only appeared in combination 
with a clustered code (table 5). 52 61 Among studies examining 
adverse drug events in general, the median number of codes 
used was 190 (IQR 156-289). Seven studies used the external 
injury cause codes Y40.0-59.9 only, 25 26 39 51 54 56 57 five 
studies used disease manifestation codes only, 23 24 32 33 37 
and 16 studies used a combination of both types of 
codes. 14 21 22 29 34 " 36 38 40 42 43 49 50 52 53 58 Only one guide- 
line recommended the use of algorithms to search for clustered 
codes, specifying external injury cause codes that should be clus- 
tered with specific disease manifestation codes in order to iden- 
tify known adverse drug events. 52 This list of clustered codes 
can be accessed freely online. 61 The most common disease 
manifestation codes used are listed in table 6. Two authors inde- 
pendently assigned causality ratings to each ICD-10 code that 
had not previously had a causality rating assigned. The k statistic 
as a measure of interrater agreement was 0.88 (95% CI 0.78 to 
0.97). 

Among studies looking for drug or drug class-specific adverse 
events, all studies reported the entire code set they used. The 
median number of codes that was used was three (IQR 1.5-10) 
(table 4). One study used external injury cause codes only, 44 six 
studies used disease manifestation codes only, 28 30 31 46 47 59 
and six studies used a combination of both types of codes 
without any requirement for the codes to be 
clustered. 27 41 45 48 55 60 



Quality assessments 

Among the 28 studies looking at adverse drug events in general, 
19 reported an explicit definition of their primary outcome 
measure that was independent of the ICD-10 code 



Table 2 


Causality rating system adapted with modifications from Stausberg and Hasford 21 22 


Code 




Examples 


category 


Definition 


Code 


Code description 


A1 


The ICD-10 code description includes the phrase 'induced by 


J70.2 


Acute drug-induced interstitial lung disorders 




medication/drug' 






A2 


The ICD-10 code description includes the phrase 'induced by medication 


142.7 


Cardiomyopathy due to drugs and other external agents 




or other causes' 


T88.7 


Unspecified adverse event of drug or medicament 


B1 


The ICD-10 code description includes the phrase 'poisoning by 


T36 


Poisoning by systemic antibiotics 




medication' 






B2 


The ICD-10 code description includes the phrase 'poisoning by or 


X44 


Accidental poisoning by, and exposure to, other and unspecified drugs, 




harmful use of medication or other causes' 




medicaments and biological substances 


C 


Adverse drug event deemed to be very likely although the ICD-10 code 


L51.2 


Toxic epidermal necrolysis 




description does not refer to a drug 






D 


Adverse drug event deemed to be likely although the ICD-10 code 


N17 


Acute renal failure with tubular necrosis 




description does not refer to a drug 






E 


Adverse drug event deemed to be possible although the ICD-10 code 


K25 


Gastric ulcer 




dictionary does not refer to a drug 






U 


Adverse drug event deemed unlikely 


I49.0 


Ventricular fibrillation and flutter 


V 


Vaccine-associated adverse event 


A80.0 


Acute paralytic poliomyelitis, vaccine-associated 


The causality ratings were modified for the purposes of this systematic review. We added category U for ICD-10 codes that have been used by others to identify adverse drug events, 


but which we felt were unlikely to be adverse drug event related. We also added category V to indicate codes that may be vaccine-related. 


ICD, International Classification of Diseases. 
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Titles Identified (4241) 

• Electronic Bibliographic Search (duplicates removed): 
o MEDLINE English(631) 

o MEDLINE Non-English (159) 

o EMBASE English (1641) 

o EMBASE Non-English (327) 

o International Pharmaceutical Abstracts (239) 

o CENTRAL (25) 

o Cochrane Database of Systematic Reviews (72) 
o Web of Science (21) 

• Handsearch of Speciality Journals & Conference Proceedings (34) 

• Electronic Grey Literature Search (134) 

• Handsearch of Bibliographic References (943) 

• Environmental Scan of the Literature (15) 

• References identified bv content experts (1) 



Excluded on title review (2875) 



Abstracts Reviewed (1366) 

• Electronic Bibliographic Search (1117) 

• Handsearch of Speciality Journals (13) 

• Electronic Grey Literature Search (74) 

• Handsearch of Bibliographic References (147) 

• Environmental Scan of the Literature (15) 

• References identified by content experts (1) 



Excluded on abstract review 



Full Texts Reviewed (340) 

• Electronic Bibliographic Search (258) 

• Handsearch of Speciality Journals (5) 

• Electronic Grey Literature Search (10) 

• Handsearch of Bibliographic References (51) 

• Environmental Scan/Content Experts (16) 





Excluded on full text review (299) 




• Did not use ICD-10 coding system (312) 




• Does not report any outcomes of interest (42) 




' • Pediatric data only (16) 


► 




• Language other than English, French or 




German (8) 




• Reported adverse events to illicit drugs (9) 




• Reported on intentional overdoses (11) 




• Duplicate records (3) 




• Could not access (1) 



Included Studies (41) 



Figure 1 Flow diagram of included studies. 



set. 14 21 22 25 29 32 3 6 - 40 42 43 49 51 54 56 " 58 Of these, 9 reported 
definitions for adverse drug reactions, 29 36 39 40 49 51 56 57 62 
5 definitions for adverse drug events, 14 21 22 38 58 an d Q ne expli- 
cit definitions for both. 54 Three studies used death as a result of 

1 • 37 42 43 

poisoning or prescription drug use as primary outcome. 
Among the 13 studies on drug or drug class-specific adverse 
events, all provided definitions for their primary outcome 



measure with 4 using drug-induced deaths 41 44 45 47 and 4 hos- 
pital admission due to an adverse drug event. 30 46 55 59 

Among the 28 studies looking at adverse drug events in 
general, 11 provided methods for their selection of ICD-10 
codes. 21 - 23 34-36 38 40 42 53 58 Methods included searching the 
ICD-10 code dictionary for diagnoses that could be attributable 
to medications (ie, gastric ulcer) and/or phrases (ie, 
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Table 3 Characteristics of 28 studies looking at adverse drug events in general, that is, events that were not specific to any drug or disease category 



Study 



Country Setting Design Data source Main objective 



Main outcome and definition 



No. 

codes 



Methods to identify ICD-10 codes 



Sample 
size 



Frequency of outcome 
measure 



Malpass ef a/ 53 


Australia 


NR 


Review 


NR 


To describe an ADE monitoring 
system 


ADE: NR 


318 1 
AM 


Mapped an adverse event 
classification system to ICD-10. 


NR 


NR 


Cox etal 26 


England 


Hospital 


RS 


Admin and PV 


To compare ADR reports in 
administrative and PV data 


ADR: NR 


175 


NR 


21 635 
records 


0.2% of admissions due 
to ADR 


Runciman ef a/ 54 


Australia 


Hospital 


Review 


Admin, trial, 
drug use, chart 
review and VS 


To review information about 
ADE and medication errors in 
Australia 


ADR: Noxious and unintended response 
to a drug used at doses for prophylaxis, 
diagnosis or therapy of disease or 
modification of function. 
ADE: ADR, harm from medication errors 
and underuse 


175 
AM 


NR 


NR 


► ADR: occur in 1 % of 
admissions. 

► ADE: occur in 2-4% 
of admissions 


Waller etal 35 


England 


Hospital 


RS 


Admin 


To describe records coded as 
drug-induced and assess their 
utility for research 


ADR: NR 


243 


Codes containing 'drug-induced', 
diagnoses 'due to' a drug, 'clearly 
implying' an ADR and Y40-59 


53.8M 
records 


0.4% of admissions due 
to ADR 


CDC 37 


USA 


NR 


RS 


vs 


To describe trends in poisoning 
deaths 


Death from ingestion, inhalation or 
exposure to pharmaceuticals, illicit drugs 
and chemicals 37 


137 


NR 


NR 


5.0-7.8 deaths/100 000 
population 


Wysowski 43 


USA 


NR 


Letter 


vs 


To study deaths attributed to 
therapeutic drug use 


Death attributed to drugs used 
therapeutically 


4 


NR 


604 
records 


NR 


Moneret et al 32 


NR 


NR 


Review 


NR 


To review the epidemio-logy of 
anaphylaxis 


Anaphylaxis 


6 


NR 


NR 


NR 


Burgess ef a/ 49 


Australia 


Hospital 


Case 
series 


Admin data 


To examine trends in 
ADR-related admissions in 
people >60 years 


ADR: Noxious and unintended response 
to a drug that occurs at doses normally 
used in humans 


200 
AM 


NR 


NR 


0.8% of admissions 
associated with ADR 


Barrow ef a/ 23 


England 


Hospital 


RS 


Admin and PV 


To compare ADR in admin data 
with PV reports 


ADR: NR 


37 


Used codes identified by Waller ef a/ 35 


NR 


NR 


Lugardon ef a/ 29 


France 


Hospital 


RS 


Admin and PV 


To estimate the incidence of 
serious ADR in hospital 


ADR: Noxious and unintended response 
to a drug used at doses for prophylaxis, 
diagnosis, or therapy of disease or 
modification of physiological function 


299 


NR 


261 

records 


2.9% of admissions 
associated with ADR 


Wysowski 42 


USA 


NR 


RS 


vs 


To identify prescription drugs 
associated with >1 000 deaths/ 
year 


Death due to a prescription drug 


NR N 


Seven disease manifestation codes 
and codes listing prescription drugs as 
cause 


NR 


NR 


Zhang ef a/ 57 


Australia 


Hospital 


RS 


Admin, VS and 


To examine trends in repeat 


Hospitalization for ADR. 


175 


NR 


37 296 


30.3% of ADR-related 










census 


ADR causing hospitalization in 
elderly 


ADR: Noxious and unintended response 
to a drug at doses normally used in 
humans 


AM 




records 


admissions were repeat 
events 


Patel efa/ 34 


England 


Hospital 


RS 


Admin 


To examine trends in hospital 
admissions associated with 
ADR 


ADR: NR 


245 


Codes containing 'drug-induced', 
indicating a diagnosis 'due to' a drug, 
and codes Y40-59 


88M 
records 


0.5% of admissions due 
to ADR 


Phillips ef al 


USA 


NA 


RS 


vs 


To describe trends in fatal 
medication errors 


ADE: Preventable deaths resulting from 
accidental overdose, wrong drug given or 
taken in error, and other accidents in the 
use of drugs 


180 


NR 


50M 

death 

records 


0.4% of deaths due to 
fatal medication errors 


Hwang ef al 58 


Korea 


Hospital 


RS 


Chart review 


To evaluate an electronic ADE 
monitoring system 


ADE: Injury from a medical intervention 
related to a drug 


326 


Codes corresponded to ADE described 
in four previous studies on ADE 
monitoring systems 73-76 


598 

patients 


31% of patients admitted 
to hospital 


Benkhaial ef a/ 24 


Germany 


Hospital 


RS 


Admin 


To assess the value of ICD-10 
codes to identify drug allergies 


Drug allergy: NR 


35 
GM 


NR 


200 
records 


9% of records indicating 
an allergy 



Continued 



Table 3 Continued 
















No. 




Sample 


Frequency of outcome 


Study 


Country 


Setting 


Design 


Data source 


Main objective 


Main outcome and definition 


codes 


Metnoas to identity ilu-iu codes 


size 


measure 


■ 

Hodgkin-son 


Australia 


Hncnital 


R<^ 


Arlmin nnrl D\/ 
MUllllll dllU rv 


To compare ADR identification 


ADR; Noxious and unintended response 


1 7R 
I / J 


MR 
IM rv 


1 1 41 A 
I Z H I *\ 


■ ■ 

4.5 /o ot admissions 


ef a/ 51 










iicinn /"nrlinn ci ih/gi anro \A/itn 
UblllLj LUUIIILj bUI Vcllldlllc Willi 


tn 3 rlri in tn^t c\rr\ ire rlncoc i icorl fn r 
LU d UlULj LlldL ULLUIb dL UUbcb UbcU IUI 


AM 




records 


3ccr\/"i3toH \A/itn Ar^R 
dbbUUdLcU VVILII f-KUW 












spontaneous reporting 


prophylaxis, diagnosis, or therapy, or 






















modification of physiological function 










\A/i i 40 

vvu 


Canada 


ED 


RS 


Admin data 


To estimate the incidence of 


ADR: Injury resulting from a medical 


245 


;34 

Used codes identified by Patel ef al 


966 232 


0.8% of ED visits were 












AnR-rolatorl FH \/icitc anrl 
RUIx IcIdlcU CU VlblLb dllU 


intorwontinn ro 3+ i n r\ tn 3 rlriin 
IIILclVcllLIUII IcIdLlliy LU d Uiliy. 






rGcords 


ADR-rolatorl 
RUIx IcIdLcU 












admissions for patients 






















>65 years 












Zlldliy cL al 


A i ictral i 3 
RUbLldlld 


Uncni+al 
nUbpi Idl 


RS 


Arlmin \/Q anrl 
MUllllll, Vj dllU 


Tn irlontifv/ f:a/~tnrc tnat nrorli/"!" 
IU lUcllLlly IdLLUIb LlldL UlcUILL 


AnR" Harmfiil nr nnn oscsnt roaz-tinn 
MUI\. nd 1 II 1 1 UI UI UlipicdbdllL IcdlLIUII 


1 75 


NR 


28 548 


17 7% nf AnR-rphtprl 
I / ./ /0 UI RUIx IcIdLcU 










census 


repeat hospital admission for 


related to a drug that predicts hazard 


AM 




patients 


admissions were repeat 












ADR in older adults 


from future use and warrants prevention, 








events 














treatment, dose change or withdrawal 
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Hohl et al 


Canada 


ED 


RS 


Admin and 


To measure proportion of 


ADE: Untoward and unintended 


650 


Adapted previously established code 


1574 


14.0% of ED visits ADE 


submitted 38 








prospect data 


ADE-related ED visits 


symptoms, signs or abnormal laboratory 




set 21 22 with others found through 


records 


related 












identifiable in admin data 


values from medication use 




literature review 






'Use of the AM modification likely, although unable to 


verify with authors. 














"Only codes associated with >1 000 deaths and/or >1 000 total mentions per year were listed. 












'"Jackson ef al describe the CHADx algorithms to identify hospital acquired diagnoses, including adverse drug events (ADE). The CHADx code set and algorithms are published on the Australian Commission on Quality and Safety in Healthcare website. 


Jackson et al. recommend searching for ADE using disease manifestation codes clustered with external injury cause codes (Y40-59). C-prefixes are codes that were introduced in the Victorian addition of the Australian Modification of ICD-10. 


lv The difference in the number of codes used by Stausberg ef al. has to do with the splitting of code E66.1 (in ICD-10-German Modification 2006) into the four codes: E66.10, E66.11, E66.12 and E66.19 (in ICD-10-German Modification 2008). 


ADE, adverse drug event; Admin, administrative; ADR, adverse drug reaction; AM, Australian modification; CDC, Centers for Disease Control and Prevention; ED, emergency department; GM, German modification; ICD, International Classification of 


Diseases; M, million; NR, not reported; pros, prospective; PV, pharmacovigilance; RS, retrospective; VS, vital statistics. 











Table 4 Characteristics of 13 studies looking at drug or disease-specific adverse drug events, in order of publication year 



Methods to Frequency of 















Main outcome and 


No. 


identify ICD-10 


Sample 


outcome measure 


Study 


Country 


Setting 


Design 


Data source 


Main objective 


definition 


codes 


codes 


size 


reported 


Gaus ef a/ 28 


Germany 


Outpatients 


RS case 
crossover 


Admin and drug use 


To illustrate case crossover methodology to 
identify ADR using bleeding complications as 
an example 


Bleeding complications 


84 


NR 


320 644 
records 


3.5 episodes of 
bleeding/1 00 years 
observation 


Wysowski 41 


USA 


NR 


RS 


Drug use and VS 


To determine the number, rate and types of 
deaths attributed to x-ray contrast media 


Death from contrast 
agents 


3 


NR 


NR 


1.1-1.2 deaths/million 
doses 


Wysowski 44 


USA 


NR 


RS 


Admin, drug use, PV 
and VS 


To compile and analyze data on the prevalence 
of bleeding related to warfarin 


Warfarin-related deaths 


1 


NR 




0.4-0.5 deaths/100 000 
population 


Sims ef a/ 47 


USA 


NR 


RS 


Admin, drug use and 
vital stats 


To examine the utility of ADR surveillance 
methods that combine and analyze multiple 
data sources 


Methadone-related death 


1 


NR 


NR 


0.8-4.3 deaths/100 000 
population 


Myers ef a/ 46 


Canada 


Hospital 


RS 


Admin and chart 
review 


To validate coding algorithms for 
acetaminophen overdose and hepatotoxicity 


Admission for 
acetaminophen toxicity 


16 


NR 


1 776 cases 


NR 


Molokhia ef a/ 31 


France 


Hospital 


RS 


Admin and PV 


To estimate the incidence and reporting rate of 
nonfatal drug-induced LQTS leading to VT and/ 
or death 


Drug-induced LQTS 


3 


NR 


861 cases 


10.9 cases/million 
population/year 


Elalamy ef a/ 27 


France 


Hospital 


RS 


Admin and 
laboratory 
surveillance 


To estimate the average cost of one episode of 
HIT in France 


HIT 


3 


NR 


50 958 
records 


0.9% of admissions 


Lyytikainen 


Finland 


Admin 


RS 


Admin and VS 


To determine the prevalence of CDAD in 


Admission associated with 


2 


NR 


NR 


16-34 cases/100 000 


2009 30 




data 






hnQnitali7Prl natipirK 
i luipi ICHIZ.CU panel 113 


CDAD 








nnni itatinn 

[JUfJU ICHIUI 1 


Li efa/ 45 


USA 


Hospital 


RS 


Admin and VS 


To examine the epidemiology of 
anesthesia-related deaths 


Anesthesia-related death 


46 v 


Lit review and 
ICD-10 search 


NR 


8.2 deaths/million 
surgical discharges 


Treeprasertsuk 


Thailand 


Hospital 


RS 


Admin 


To examine the incidence and complications of 


Drug-induced liver injury 


4 


NR 


237 970 


0.03% of admitted 


efa/ 60 










antimicrobial induced liver injury in hospitalized 
patients 








records 


patients 


Rhee efa/ 59 


Korea 


Hospital 


RS case 
control 


Admin 


To quantify the risk of digoxin toxicity with 
concomitant use of diuretics 


Admission for digoxin 
toxicity 


1 


NR 


104 075 
records 


61.5 cases/100 000 
admissions 


Sood ef a/ 55 


Australia 


Hospital 


RS 


Admin 


To examine the epidemiology, outcomes and 
burden of acetaminophen poisoning 


Admission for 
acetaminophen poisoning 


2 

AM 


NR 


NR 


39-46 cases/100 000 
admissions 


Wysowski ef a/ 48 


USA 


NR 


RS 


PV, admin, VS, drug 
use and surveillance 


To determine the incidence of serious 
anaphylactic reactions to parenteral iron 


Anaphylaxis due to 
parenteral iron 


2 vi 


NR 


NR 


0.1-0.3 deaths/million 
doses sold 



includes one code unrelated to ADE, ADR (eg, Y65.3 Endotracheal tube wrongly placed). 

vl The authors used surveillance data from the National Electronic Injury Surveillance System-Cooperative Adverse Drug Event Surveillance (NEISS-CADES) and the Drug Abuse Warning Network (DAWN) Live!. Codes indicating anesthesia-related events that 
were not medication-relation have been omitted. 

Admin, administrative; ADR, adverse drug reaction; AM, Australian modification; CDAD, Clostridium difficile-associated disease; ED, emergency department; HITS, heparin-induced thrombocytopenia; ICD, International Classification of Diseases; LQTS, long 
QT syndrome; NR, not reported; PV, pharmacovigilance; RS, retrospective; VS, vital statistics; VT, ventricular tachycardia. 
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Table 5 Disease manifestation codes that were used only as 
clustered codes and never as stand-alone codes 



ICD-10 code 


Description 


D68.8 


Other specified coagulation defects 


F05 


Delirium, not induced by alcohol and other 




psychoactive substances 


195 


Hypotension 


L21 


Seborrhoeic dermatitis 


L26 


Exfoliative dermatitis 


L27 


Dermatitis due to substances taken internally 


L28 


Lichen simplex chronicus and prurigo 


L30 


Other dermatitis 


R20 


Disturbances of skin sensation 


R23 


Other skin changes 


R40 


Somnolence, stupor and coma 


R41 


Other symptoms and signs involving cognitive 




functions and awareness 


R44 


Other symptoms and signs involving general sensations 




and perceptions 



ICD, International Classification of Diseases. 



'drug-induced'), empiric study of existing pharmacovigilance 
reports and mapping of events to the ICD-10 code diction- 
ary, 21 22 53 and literature review. 21 22 34-36 Others adopted code 
sets that had previously been established by other 
authors. 23 38 40 58 

Only two studies estimated the sensitivity with which their 
code set may have ascertained the desired outcomes. 38 40 Wu 
et al 40 compared records containing adverse drug event-related 
diagnoses between two administrative databases. The authors' 
premise was that for patients admitted to hospital through the 
emergency department for a drug-related diagnosis, the emer- 
gency department discharge and hospital admitting diagnoses 
should be similar. Using an ICD-10 code set containing 245 
drug-related codes, including the external injury cause codes, 
Wu et al 40 found that 15% of drug-related emergency 



Table 6 Top 1 5 disease manifestation codes used to identify 
adverse drug events in all studies 

ICD-10 code Description 



T88.6 


Anaphylactic shock due to the adverse effect of a drug 


T88.7 


Unspecified adverse effect of a drug 


N14.1 


Nephropathy induced by drugs, medicaments and biological 




substances 


D59.0 


Drug-induced autoimmune hemolytic anemia 


D59.2 


Drug-induced non-autoimmune hemolytic anemia 


D61.1 


Drug-induced aplastic anemia 


J70.4 


Drug-induced interstitial lung disorders 


K71 


Toxic liver disease with cholestasis 


K71.1 


Toxic liver disease with hepatic necrosis 


K71.2 


Toxic liver disease with acute hepatitis 


K71.6 


Toxic liver disease with hepatitis, not elsewhere classified 


K71.9 


Toxic liver disease, unspecified 


L56.1 


Drug photoallergy response 


N14.2 


Nephropathy induced by unspecified drug, medicament or 




biological substance 


T88.3 


Malignant hyperthermia due to anesthesia 



ICD, International Classification of Diseases. 



department visits leading to hospital admission were coded with 
a corresponding admitting diagnosis. Wu et al 40 estimated the 
specificity of their code set, and found it to be 99.7%. In com- 
parison, in a study including both admitted and discharged 
emergency department patients, in which adverse drug events 
were identified prospectively by pharmacists and physicians, 
6.8% of prospectively identified adverse drug events were iden- 
tifiable using an ICD-10 code set consisting of diagnoses rated 
as definitely, very likely or likely to be related to medications. 38 
When the code set was broadened to include lower likelihood 
codes, 28.1% were identifiable with little drop in the code set's 
specificity (98.7-87.7%). 

Three studies considered the uncertainty of the causal link 
between drug exposure and the adverse event, and provided 
analyses allowing the reader to ascertain the impact that this 
may have had on the study results. 21 22 38 

DISCUSSION 

Our objective was to synthesize a comprehensive set of ICD-10 
codes and coding algorithms that have been used by health 
researchers to identify adverse drug events in administrative 
health data. Among 41 published studies, we found 827 ICD-10 
codes that have been used for this purpose. There was a large 
degree of variability in the number and types of codes used 
between studies, and only one published guideline recom- 
mended the use of algorithms to identify external injury cause 
codes clustered with disease manifestation codes. Of the 
reviewed studies, two provided estimates of the code set's sensi- 
tivity and specificity. 

Adverse drug events represent a growing public health 
concern. 3 In the USA, adverse drug events represent the fourth 
to sixth leading cause of death, and are a frequent cause of 
unplanned hospitalizations, emergency department visits and 
ambulatory care encounters. 5 10 63-65 The focus of the US$1 
billion federal private-public initiative, Partnership for Patients, 
is to reduce hospital-acquired conditions by 40% and hospital 
readmissions by 20% by the end of 20 13. 66 

In order to accomplish this target, the Partnership for Patients 
has identified the reduction of in-hospital adverse drug events as 
a priority. One intervention that the Partnership for Patients is 
promoting to accomplish this goal is medication reconciliation, 
a health systems intervention aimed at decreasing adverse drug 
events that result from the inaccurate transfer of medication 
information. 67 68 However, to date little research has been con- 
ducted to describe and rank possible etiologies of adverse drug 
events, and as a result, it is largely unknown to what extent 
inaccurate transfer of medication information contributes to the 
development of clinically significant adverse drug events. Thus, 
it is not surprising that a recent systematic review of 26 con- 
trolled studies failed to find an effect of medication reconcili- 
ation on downstream health services use, mortality or cost. 69 
This example underscores the need for further development of 
innovative, evidence-based and effective patient safety strategies 
to reduce adverse drug events, and for their evaluation on 
health outcomes before their implementation outside of the 
research setting. 

In order to inform the development of strategies to reduce 
preventable adverse drug events, the burden of disease in differ- 
ent healthcare settings and patient populations and their 
common etiologies need to be understood. This will help to pri- 
oritize and rationalize the development and evaluation of emer- 
ging strategies to prevent commonly occurring events associated 
with health services use and cost. Modifiable risk factors that 
can be targeted in carefully designed health systems 
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interventions need to be identified. These may be related to 
medications, medication classes, treatment protocols, prescribing 
patterns, patient groups, provider groups, healthcare settings, 
and models of care, some of which may assist in developing suc- 
cessful interventions. Once developed, strategies are likely to 
benefit from refinement to enhance their feasibility of imple- 
mentation and their performance. Finally, their impact on 
health outcomes and cost must be evaluated and compared with 
that of other health interventions in order to guide rational 
resource allocation and optimize health value for expenditure. 

Population-level administrative health data that can be linked 
with medication dispensing data may represent a rich source of 
health information for this type of work. Adverse drug event 
data from this source may offer accessible and standardized 
population-level data over long time periods, enabling analysis 
of time trends, prescribing patterns, and comparisons across 
healthcare settings. 16 70 However, no consensus presently exists 
among health researchers on how to identify adverse drug 
events reliably within such data sources, leading to substantial 
variability in the methods used for their identification. 

Our study is the first systematically to review the health litera- 
ture to synthesize a comprehensive set of ICD-10 codes previ- 
ously used to identify adverse drug events. Previous studies have 
identified code sets by relying on ad hoc reviews of the litera- 
ture, and mapping of drug-related diagnoses and pharmacovigi- 
lance case reports to the ICD-10 code dictionary. Most have 
adopted and used code sets developed by previous authors 
without conducting any validation studies to understand their 
sensitivity or specificity. When examining the code sets, 
common manifestations of adverse drug events have often been 
omitted (eg, El 6.2 hypoglycemia), while the codes of rare 
events are commonly used (eg, T88.3 malignant hyperthermia 
due to anesthesia). This is problematic, as multiple studies relied 
entirely on disease manifestation codes to identify drug-related 
diagnoses. The omission of common manifestations of adverse 
drug events from their code sets based on the assumption that 
they might be associated with low positive predictive values 
would probably have dramatically influenced the numbers and 
types of adverse drug events found. 

There is general agreement among health researchers that 
adverse drug events are underreported in administrative data, 
and that the effect of coding quality on adverse drug event iden- 
tification is poorly understood. 23 34 38 ^° 71 Based on our 
review, it is also possible that the use of incomplete code sets 
for adverse drug events may be a contributing factor. We found 
only two studies that evaluated the sensitivity of their ICD-10 
code sets for adverse drug events, and both were low. 38 40 
Therefore, validation of a more comprehensive set of adverse 
drug event-related ICD-10 codes is necessary to try and 
enhance the sensitivity of the code sets used, while retaining 
specificity. This work needs to be conducted in a variety of care 
settings (eg, hospital vs ambulatory care), on a variety of 
adverse drug event types (eg, adverse drug reactions vs non- 
adherence), on different grades of adverse drug event severity 
(eg, severe vs mild), and by syndrome (eg, intracerebral hemor- 
rhage vs epistaxis). Different clinical practice settings may influ- 
ence the diagnostic performance of the code set(s) that is/are 
used, and may require refinements of the code sets used. Finally, 
it may be that administrative data may be well suited to tracking 
and investigating some consistently coded and identifiable 
adverse outcomes (eg, bleeding events), but not all manifestations 
of drug-related events (eg, delirium). Thus, we cannot recom- 
mend the adoption of our proposed code set without validation 
and further refinement. Instead, we present a comprehensive list 



of codes that we hope will provide the basis for further investiga- 
tion, debate and consensus building in this area. 

Due to the multiple ways in which adverse drug events may 
be coded (ie, by using external injury cause codes only, disease 
manifestation codes only, clusters of codes, or a combination of 
these methods), methodologies need to be developed to avoid 
double counting. Two of the studies we reviewed concluded that 
double counting was indeed possible when searching for adverse 
drug events using a combination of external injury cause codes 
and disease manifestation codes, and that this occurred in up to 
15% of records. 35 36 Similarly, studies need to be conducted to 
understand to what extent the use of disease manifestation 
codes (ie, E87.1 hyponatremia), which may indicate an adverse 
drug event or a non-drug-related event, may influence the sensi- 
tivity and specificity of the code set. To date, only one study has 
compared the sensitivity and specificity of narrower and broader 
code sets, and compared them to an independent prospective 
criterion standard in emergency department administrative 
data. 38 In that study, while the broader code set led to higher 
sensitivity (6.8% vs 28.1%), broadening the code set had little 
impact on the code sets' specificity. 38 Unfortunately, the study 
did not examine coding quality to determine which steps during 
the patient care and coding trajectory may have contributed 
most to the under-coding of adverse drug events. Finally, 
methods to identify and understand the extent to which adverse 
events related to prescription medication use may be coded 
using codes that do not distinguish between prescription drugs 
and drugs of abuse (eg, Fll mental and behavioral disorders 
due to the use of opioids) need to be developed. 

The most widely used definition for adverse drug events is 
'harm caused by the use of a drug'. 2 72 In this study, we pre- 
sented all definitions as reported by the study authors, as these 
may have led to variability in the code sets used. The existing 
inconsistency in the operational definition of adverse drug 
events needs to be addressed before being able to achieve con- 
sensus on a common code set(s), and may enhance the consist- 
ency with which adverse drug events are identified and 
reported, and thus comparability between studies. Given this 
limitation, we provided a comprehensive list of definitions and a 
corresponding code set that may serve as a point of reference 
for consensus building. 

We did not attempt to meta-analyze data on the prevalence of 
adverse drug events, as this was not the objective of our study. 
In addition, significant differences in the ICD-10 code sets used 
to find adverse drug events are likely to result in significant het- 
erogeneity between studies, and any differences that are found 
may simply be due to the methods used to identify them. 

Ongoing national adaptations of the ICD-10 coding systems 
have introduced additional variability in the coding of adverse 
drug events that we were unable to account for. Not all studies 
described explicitly which national adaptation and versions of 
the coding dictionary they based their code set on. Some adap- 
tations, for example, the German modification, may use add- 
itional two-decimal subcategorizations of individual disease 
manifestation codes that allow more refined coding than other 
systems. At present, the USA only uses ICD-10 coding for mor- 
tality reporting, explaining why all the US studies reported only 
on events related to death. Thus, while the majority of code cat- 
egories are comparable across coding systems, the variability 
between national adaptations and coding versions used needs to 
be taken into account before application of any code set. 

Additional limitations of our work are that we only reviewed 
publications in English, French and German. We also did not 
search extensively for abstracts or conference proceedings, as we 
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thought we would be unlikely to find code sets published in 
these formats. We applied the only causality rating system that 
we are aware of for adverse drug event-related ICD-10 
codes. 21 22 While the causality rating system is based on clinical 
reasoning, and therefore inherently subjective, it may provide 
health researchers with a framework with which to start incorp- 
orating the certainty/uncertainty of drug-related causes to diag- 
noses identified in ICD-10. Therefore, we applied the 
previously proposed causality categories to additional ICD-10 
codes that we identified through our review. 

In conclusion, in this study we have synthesized a set of 
ICD-10 codes that have been used by health researchers to iden- 
tify adverse drug events in administrative health data. Our code 
provides a basis for future work in establishing comprehensive 
and agreed-upon code sets that can be validated and refined for 
future work in this area. 
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